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Cross Reference to Related Application 

[0001] This application is related to U.S. Patent Application No. (Attorney 

10 Docket No. SONYP024/SCEA02019US00), filed on the same day as the instant 
application and entitled "METHOD AND APPARATUS FOR REAL TIME GLOBAL 
ILLUMINATION INCORPORATING STREAM PROCESSOR BASED HYBRID 
RAY TRACING," which is incorporated herein by reference for all purposes. 

Background of the Invention 

15 1. Field of the Invention 

[0002] This invention relates generally to video processing and more particularly to 
providing real-time shading model for rendering objects on a display. 

2. Description of the Related Art 

[0003] Game developers are constantly refining image synthesis to create a technique 
20 that presents a realistic visual impression for users. One aspect of image synthesis that is 
used to present a realistic visual impression is the accurate portrayal of shadows and 
shading. Normal lighting for computer games rely on vector dot product calculations, 
which are relatively simple, but inaccurate. While the vector dot product calculations are 
able to present hard edge shadows, theses type of calculations cannot present soft edge 
25 shadows. 
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[0004] Figure 1 is a schematic diagram representing an exemplary lighting computation 
scheme for rendering a display. Light source 100 includes a number of points that emit 
light. Occlusion surface 102 blocks a portion of the light from light source 100 that is 
directed to surface 104. Thus, a hard edge shadow (the umbra) is defined by region 106 
from occlusion surface 102. However, point 110, which lies in soft shadow (or 
penumbra) region 108, requires that multiple points on light source 100 be picked out and 
the shadow projection for each of the points is determined. Each of the shadow 
projections is then summed, therefore, with respect to point 110 the summation of a 
number of lighting functions is required to realistically display points in soft shadow 
region 108. However, the calculation and summation of the lighting functions are 
computationally intensive and not viable alternatives for presenting realistic shading in 
real time. Furthermore, adding another light source, which is common especially in 
video games, requires another complete calculation. 

[0005] One attempt to address this shortcoming is the use of a special case list which 
may be used to define the shading properties when certain criteria are met. However, the 
special case list is not exhaustive and as a result the list must be updated for each new 
situation. Moreover, the attempt to force a particular situation to fit a special case may 
result in sub-optimal display quality. Additionally, the special case list acts as a 
restriction on the game developer. 

[0006] As a result, there is a need to solve the problems of the prior art to provide a 
method and apparatus for providing a shading scheme capable of rendering a realistic 
representation in real time. 
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Summary of the Invention 



[0007] Broadly speaking, the present invention fills these needs by providing a method 
and apparatus that simplifies the lighting function through the use of omni-directional 
5 basis functions, and in combination with texture mapping, provides for the real time 
display of realistic shading with minimal computations. It should be appreciated that the 
present invention can be implemented in numerous ways, including as a method, a 
system, computer readable medium or a device. Several inventive embodiments of the 
present invention are described below. 

10 [0008] In one embodiment, a method for rendering an image with high resolution 
lighting characteristics is provided. The method initiates with generating a texture map 
associated with the image. The texture map is defined by texels. Then, a value 
representing a lighting characteristic for each of the texels is calculated. Next, the value 
is stored. Then, a coordinate space of the texture map is associated with a display screen 

15 coordinate space. Next, the image is rendered on a display screen using the stored value. 
[0009] In another embodiment, a method for incorporating lighting characteristics of an 
image of an object into a texture map is provided. The method initiates with defining a 
texture map associated with the image. Next, a lighting characteristic associated with a 
texel of the texture map is determined. Then, the texel is associated with the lighting 

20 characteristic. 

[0010] In yet another embodiment, a method for rendering an image is provided. The 
method initiates with defining a texture map associated with the image. Then, a value 
corresponding to a multi-directional signal is associated with a texel of the texture map. 
Next, an intensity of a pixel associated with the texel is determined. When determining 
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the intensity, the method includes accessing the value associated with the texel, and 
applying the value to a quantity representing a light source component. 
[0011] In still yet another embodiment, a computer readable medium having program 
instructions for rendering an image with high resolution lighting characteristics is 
5 provided. The computer readable medium includes program instructions for accessing a 
lighting characteristic value associated with a texel of a texture map associated with the 
image. Program instructions for associating a coordinate space of the texture map with a 
display screen coordinate space are included. Program instructions for applying the 
lighting characteristic value to a corresponding pixel for presentation on the display 

10 screen are also provided. 

[0012] In another embodiment, a computer readable medium having program instructions 
for incorporating lighting characteristics of an image of an object into a texture map is 
provided. The computer readable medium includes program instructions for defining a 
texture map associated with the image. Program instructions for determining a lighting 

15 characteristic associated with a texel of the texture map are provided. Program 
instructions for associating the texel with the lighting characteristic are included. 
[0013] In yet another embodiment, a computing device is provided. The computing 
device includes a memory capable of storing data representing a texture map associated 
with an image. The texture map contains a texel. The texel is associated with data 

20 describing a light field for a point within the texel according to a basis function. The 
computing device includes logic for mapping the texel to a pixel associated with a display 
screen in communication with the computing device. Logic for accessing the data 
describing the light field is provided. Logic for determining an intensity associated with 
the pixel based upon the data describing the light field and logic for enabling presentation 

25 of the intensity of the pixel on the display screen are included. 
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[0014] An integrated circuit is provided. The integrated circuit includes a memory 
capable of storing data corresponding to a self shadow and self interreflection lighting 
characteristics of an image. Circuitry for accessing the data is provided. Circuitry for 
determining an intensity associated with a pixel based upon a product of the data and an 
5 illumination value is included. Circuitry for enabling presentation of the intensity of the 
pixel on the display screen is included. 

[0015] Other aspects and advantages of the invention will become apparent from the 
following detailed description, taken in conjunction with the accompanying drawings, 
illustrating by way of example the principles of the invention. 

10 
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Brief Description of the Drawings 

[0016] The present invention will be readily understood by the following detailed 
description in conjunction with the accompanying drawings, and like reference numerals 
5 designate like structural elements. 

[0017] Figure 1 is a schematic diagram representing an exemplary lighting computation 
scheme for rendering a display. 

[0018] Figure 2A is a simplified schematic diagram illustrating three components of a 
basis function used for eventual derivation of a lighting function in accordance with one 

10 embodiment of the invention. 

[0019] Figures 2B-2G are simplified schematic diagrams for alternatively representing 
the components of a basis function for use in deriving a lighting function. 
[0020] Figure 3 is a schematic diagram illustrating a polygon wherein ray tracing 
techniques are used to define a lighting function at points on a surface of the polygon in 

15 accordance with one embodiment of the invention. 

[0021] Figure 4A is a simplified schematic diagram of the use of Gouraud shading. 
[0022] Figure 4B illustrates a schematic diagram illustrating the process of linear 
interpolation. 

[0023] Figure 5 is a schematic diagram illustrative of a scheme for embedding lighting 
20 coefficients into a texture map in accordance with one embodiment of the invention. 

[0024] Figure 6 is a schematic diagram illustrating the interrelationship between the two 
dimensional and three dimensional coordinate spaces encountered for the real-time 
shading display of an object in accordance with one embodiment of the invention. 
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[0025] Figure 7 is a schematic diagram illustrating an alternative representation of the 
three coordinate spaces of Figures 5 and 6. 

[0026] Figure 8 is a simplified schematic diagram of the computing device capable of 
providing real time self-shadowing and self-interreflection for objects of an image in 
5 accordance with one embodiment of the invention. 

[0027] Figure 9 is a flow chart diagram of the method operations for rendering a low 
resolution image with high resolution lighting characteristics in real-time in accordance 
with one embodiment of the invention. 

[0028] Figure 10 is a schematic diagram illustrating an on-line gaming application in 
10 accordance with one embodiment of the invention. 
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Detailed Description of the Preferred Embodiments 



[0029] An invention is described for a system, apparatus and method that enables real 
time presentation of soft shadows for a video presentation, such as a video game, with 
minimal computationally intensive calculations. It will be obvious, however, to one 
skilled in the art, that the present invention may be practiced without some or all of these 
specific details. In other instances, well known process operations have not been 
described in detail in order not to unnecessarily obscure the present invention. Figure 1 is 
described in the "Background of the Invention" section. 

[0030] The embodiments of the present invention provide a system and method for 
allowing real time presentation of realistic shadows during a video presentation, such as 
in a video game. In one embodiment, spherical harmonics, a mathematical system 
analogous to the Fourier transform but defined across the surface of a sphere simplify the 
rendering equation into a dot product over the spherical harmonic coefficients. In one 
embodiment, a representation of the incoming illumination is multiplied by a 
representation of surface reflectance, referred to herein as the transfer function (T). It 
will be apparent to one skilled in the art that the orthogonality of spherical harmonic 
functions provide that the integral of the function's products is the same as the dot 
product of functions respective coefficients. Therefore, the lighting function is collapsed 
to a dot product of the coefficients of a luminance function (also referred to as an 
illumination function) describing the incoming light from all directions and the 
coefficients of a transfer function describing the surface reflectance. The transfer 
function when dotted with luminance function, i.e., multiplied and integrated, yields the 
approximated lighting for a respective point. It should be appreciated that different types 
of transfer functions may be generated for diffuse surfaces, wherein the different types of 
transfer functions are associated with varying degrees of calculation complexity. Further 
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information related to the different types of transfer functions and spherical harmonic 
lighting in general may be found in the article entitled "Spherical Harmonic Lighting: 
The Gritty Details", Robin Green, January 16, 2003, which is hereby incorporated by 
reference for all purposes. It should be appreciated that the transfer and illumination 
functions described herein are capable of being defined over all directions using a 
parameterization of directions. The parameterization of directions may be spherical, 
cubical, cylindrical, etc. Additionally, the transfer and illumination functions may be 
expressed as the sum of any suitable basis functions. Exemplary basis functions include 
spherical harmonic basis functions, wavelet, lapped basis functions, etc., wherein each of 
these basis functions are orthogonal, orthonormal, or otherwise. 

[0031] In one embodiment of the invention, the data captured through the transfer 
function is stored in memory along with texture map data. In one embodiment, a lighting 
basis function is sampled at the center of a texel and the resulting coefficient is stored, as 
opposed to calculating the lighting function at the corners of triangles. Thus, the need for 
interpolating from triangle vertices is eliminated. As will be explained in more detail 
below, a high resolution object is used to work out where the shadows are defined and the 
nature of the intensity of the shadows. That is, the self shadowing and the self 
interreflection properties of the image are captured through the transfer function. The use 
of a high resolution for the determination of the shadows allows for the data derived from 
the high polygon model to be stored and applied to a low polygon model in order to give 
a high resolution appearance to the low resolution image. In one embodiment, a one to 
one mapping between points in three dimensional space to points in two dimensional 
space enable a look-up operation for determining the corresponding pixel location 
associated with the stored transfer function data. 
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[0032] Figure 2A is a simplified schematic diagram illustrating three components of a 
basis function used for eventual derivation of a lighting function in accordance with one 
embodiment of the invention. The three components represent the illumination function 
(L), the occlusion function (O), and a cosine term (cos(0)). It should be appreciated that 
surfaces 112a, 112b, and 112c each represent a surface of a corresponding globe which 
has been "unwrapped." The illumination function (L) component determines the amount 
incoming light to a particular point from light source 114. It should be further 
appreciated that surface 112a may be referred to as a theta to phi parameterization. The 
occlusion function (O) represents an area of the light source that is blocked, i.e., occluded 
from the point. Shadow area 116 illustrates the area that is blocked or occluded from the 
light source. The third term of the spherical function is the cosine term. The cosine term 
represents the ratio of light reflected off of a surface when illuminated from a specific 
direction. The cosine term is non-zero over the upper hemisphere of sphere 112c relative 
to the surface normal. The occlusion component and the cosine component may be 
combined to define the transfer function (T). The transfer function describes the lighting 
characteristics related to self transfer and self shadowing for a particular point on an 
object. 

[0033] One skilled in the art will appreciate that spherical harmonics based functions are 
related to orthogonal polynomials, thereby allowing the representation of a function to be 
broken down to discrete polynomials through the summation of the terms of an equation. 
Furthermore, spherical harmonics based functions are rotationally invariant. That is, with 
reference to video games, using spherical harmonic based functions guarantee that during 
the animation of scenes, moving lights and rotating objects, the intensity of lighting will 
not fluctuate, crawl, pulse or have any other objectionable artifacts associated with other 
shading techniques. Furthermore, the addition of a light source does not require another 
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complete calculation when utilizing spherical harmonics based functions for determining 
the lighting function. 

[0034] As mentioned above, the transfer function (T), which includes the occlusion 
component and the cosine component, represents a description of the surface reflectance. 
The integration of the illumination function (L) and the transfer function (T) over an 
entire sphere may be simplified to a sum of dot products of their coefficients through the 
application of orthogonal basis functions. Accordingly, the lighting function may be 
collapsed into the multiplication of two different functions based on the rendering 
equation. Thus, a series of multiplication and addition operations resulting from this 
collapse of the lighting function enables for a relatively fast operation in the conversion 
of the integration over a sphere to determine lighting characteristics for an object being 
rendered. Consequently, the real-time presentation of realistic shading, as defined by the 
transfer function, may be implemented into a video game. Additionally, the basis 
functions provide soft shadows with a minimal amount of work and thereafter hard 
shadows may be defined through further calculation. Traditional approaches provide 
hard shadows initially and then require complex calculations to define the soft shadows. 
Further information on the spherical harmonic lighting technique, and spherical basis 
functions in general, is disclosed in the article entitled "Spherical Harmonic Lighting: 
The Gritty Details", Robin Green, January 16, 2003, which has been incorporated by 
reference for all purposes. It will be apparent to one skilled in the art that the invention 
described herein may be utilized with any suitable basis function and is not limited to the 
spherical harmonic functions described herein. For example, a wavelet decomposition 
may be used as well as spherical basis functions. It will be apparent to one skilled in the 
art that a wavelet is an additional way of representing a signal as a sum of frequencies. 
The basis functions may be defined across any parameterization of the directional 
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domain, for example, a spherical, a hemispherical, a cubical, a conical or a cylindrical 
parameterization. Additionally, these basis functions may be orthogonal, orthonormal, 
lapped or otherwise. 

[0035] Figures 2B-2G are simplified schematic diagrams for alternatively representing 
5 the components of a basis function for use in deriving a lighting function. Here, Figures 
2B-2G provide an alternative representation to Figure 2A. Figure 2B includes light 
source 100, occlusion surface 102 blocks a portion of the light thereby defining umbra 
106 on surface 104. Thus, in Figure 2B there are no soft shadows defined. Figure 2C 
represents a point on surface 104 being lit, as represented through an unwrapped 

10 spherical function. Figure 2D represents the lighting function (L) component of the 
spherical function. Figure 2E represents the visibility function, also referred to as the 
occlusion function (O). Figure 2F represents the cosine term (cos(0)). It should be 
appreciated that the basis function for the lighting characteristics may be expressed as the 
product of the lighting function represented by Figure 2D, the visibility function 

15 represented by Figure 2E, and the cosine term represented by Figure 2F. Figure 2G 
represents the transfer function. One skilled in the art will appreciate that the transfer 
function representation of Figure 2G excludes the lighting function of Figure 2D, i.e., the 
transfer function is a combination of the visibility function of Figure 2E and the cosine 
term of Figure 2F. 

20 [0036] Figure 3 is a schematic diagram illustrating a polygon wherein ray tracing 
techniques are used to define a lighting function at points on a surface of the polygon in 
accordance with one embodiment of the invention. Point P 124 is on a surface of 
polygon 120. The normal 122 to P is also shown. Here, rays are traced from a point, 
e.g., point P 124, through the polygon database in order to define any hits. For example, 

25 ray 126 is traced from point P 124 and will eventually hit a side of polygon 120 and 
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reflect as illustrated by ray 130. On the other hand, ray 128 does not hit any sides of 
polygon 120, i.e., is visible to the outside world. Arc 132 represents the amount of the 
outside world that point P can see. The ray tracing information distributes light, i.e., self- 
transfer. The self-transfer, also referred to as self-interreflection, is determined by the 
rays representing light being bounced off the surfaces of polygon 120 to further 
illuminate the region. In addition, it should be appreciated that with each bounce off of a 
wall of the polygon the associated light ray will lose a certain amount of energy. Further 
information on ray tracing techniques may be found in the article entitled "Spherical 
Harmonic Lighting: The Gritty Details" referenced above. 

[0037] Figure 4A is a simplified schematic diagram of the use of Gouraud shading. As is 
known, Gouraud shading uses a number of triangles in order to smoothly interpolate the 
shading of a region. Here, region 140 is shown with a plurality of small triangles in order 
to more accurately capture the shading in the region. However, region 142 consists of 
larger triangles where it is not necessary to capture the amount of shading to a great detail 
in that area. In order to provide an accurate shading picture, Gouraud shading requires 
that many small triangles be defined in the particular area. The lighting characteristics at 
each of the corners of the triangles are calculated and then through linear interpolation the 
shading for the entire triangle is determined. 

[0038] Figure 4B illustrates a schematic diagram illustrating the process of linear 
interpolation. The comers of triangle 140a may be associated with values representing 
the lighting at each of the corners of triangle 140a. Thus, in one corner, the value of 
eighty is represented while in another corner a value of fifteen, and in a third corner, a 
value of five is represented. Through linear interpolation the shading of the entire 
triangle is then determined by the relative weights of each corner, i.e., a point closer to 
the corner having a weight of eighty will have a value more closely related to eighty. It 
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should be appreciated that the computation of the polygons, i.e., triangles, is intensive 
and precludes the real-time presentation shading for video games due to the computations 
required for presenting soft shadows. Furthermore, when a light source is added or 
changed, the entire calculation must be repeated in order to render the image. 
[0039] Figure 5 is a schematic diagram illustrative of a scheme for embedding lighting 
coefficients into a texture map in accordance with one embodiment of the invention. 
Object 150a, or more specifically, the points that make up object 150a, are projected from 
world space, also referred to as 3-dimensional space, to texture map 154. For example, 
region 152a of object 150a is mapped from three dimensional space to texture map 154 
coordinate space, which is generally represented as (u,v) coordinates. Thus, region 152b 
in texture map space corresponds to region 152a in three dimensional space. The (u,v) 
coordinates of the texture map are correlated to screen space for rendering object 150b on 
a display screen. In one embodiment, the texels, which define texture map 154, include 
data representing the lighting characteristics associated with the texel. In another 
embodiment, the spherical harmonic lighting function, as described in the article entitled 
"Spherical Harmonic Lighting: The Gritty Details," which has been incorporated by 
reference, is sampled at the center of each texel and the resulting coefficients or values 
are stored in memory. Thus, for each texel, the occlusion and cosine functions, which 
make up the transfer function (T), may be cached in memory of a computing device. 
Consequently, the illumination source (L) may be determined and the lighting function 
becomes a dot product of the cached transfer function (T) value and the illumination 
source (L). With respect to video games the computing device may be a video game 
console, e.g., the "PLAYSTATION 2"® manufactured by Sony Computer Entertainment 
Inc. 
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[0040] It should be appreciated that there is no need to calculate polygons as is required 
with traditional shading schemes such as Gouraud shading. Through the application of 
spherical basis functions, such as spherical harmonic functions, the speed of rendering 
become independent of the size and number of light structures. In particular, with respect 
to video games, light sources are constantly changing. Accordingly, a value for the 
illumination function (L), representing the light source(s) is readily determined and 
combined with the stored transfer function value to provide real-time shading for an 
application. That is, the transfer function (T) value remains constant and is calculated 
once and stored. It should be appreciated that the stored transfer function value 
corresponds to a multi-directional signal which may be defined through ray-tracing in one 
embodiment. 

[0041] Figure 6 is a schematic diagram illustrating the interrelationship between the two 
dimensional and three dimensional coordinate spaces encountered for the real-time 
shading display of an object in accordance with one embodiment of the invention. As 
mentioned above, the lighting characteristic, i.e., shadow properties, for an image are 
enabled to be presented in real time by inserting lighting function data in each texel of a 
texture map. The lighting function data captures the self shadowing and the self 
interreflection characteristics through the application of the transfer function (T). In one 
embodiment, a high resolution rendition of an object is used to capture the lighting 
function data. The lighting function data is then stored and applied to a low resolution 
model. The low resolution model is made to appear as having a high resolution when the 
lighting function data is applied to the low resolution model. 

[0042] Still referring to Figure 6, object 142 is a high resolution rendition in three 
dimensional space. In order to provide the high resolution, a large number of polygons 
142, e.g., thousands of polygons, are defined on object 146. A one-to-one mapping of the 
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points in three dimensional space to points in two dimensional space is performed. Thus, 
the texels defined by texture map 148 correlate to the points on object 142 through this 
mapping. For example, texel 142-2 may correspond to polygon 142. As is generally 
known, texturing may be analogized to wrapping a three dimensional object in two 
5 dimensional gift paper. As discussed above with reference to Figure 5, texel 142-2 is 
associated with data representing a lighting characteristic, e.g., transfer function, derived 
from a spherical basis function. Thus, when rendering a two dimensional image 144 of 
three dimensional object 146 in screen space, a low resolution image is given the 
appearance of a high resolution image. That is, data representing the lighting 

10 characteristics of a high resolution image defined by thousands of polygons are captured 
by texture map 148. The captured lighting characteristics are then projected from texture 
map 148 to two dimensional image 144 that is presented on a display screen. As can be 
seen in Figure 6, image 144 is a low resolution image defined by 12 polygons (6 
polygons being shown). However, the lighting characteristics, e.g., shading, shadows, 

15 etc., for image 144 are derived from image 146 defined by thousands of polygons. 
Accordingly, these lighting characteristics may be projected onto a low resolution image 
to provide a high resolution appearance. 

[0043] Figure 7 is a schematic diagram illustrating an alternative representation of the 
three coordinate spaces of Figures 5 and 6. Texture map 154 includes region 157-1 

20 defined by a plurality of texels. Each texel may include lighting characteristic data for 
the corresponding texel, wherein the lighting characteristic data is retrievable, i.e., stored 
in memory with the texel data. For example, point 156-1 represents lighting 
characteristic data described above that is embedded into the corresponding texel. The 
lighting characteristic data is derived from the application of a spherical basis function to 

25 image 155. As discussed above with reference to Figure 6, image 155 of Figure 7 may be 

SONYP025/MLG 16 Patent Application 



a high resolution image, i.e., defined by numerous polygons. Region 157-1 of texture 
map 154 corresponds to region 157-2 of three dimensional image 155. In addition, the 
lighting characteristic data represented by point 156-1 is derived from application of a 
spherical basis function to determine the self shadowing, i.e., the occlusion function, and 
5 the self interreflection. It will be apparent to one skilled in the art that the occlusion 
function determines the visibility, i.e., self-shadowing for a respective point P. The self 
interreflection data is calculated through ray tracing in one embodiment of the invention. 
When rendering image 157-3 on a display screen, a look-up of the lighting characteristic 
data, e.g., the transfer function data, is performed. A dot product between the transfer 
10 function (T) data and the illumination function (L) data is calculated to arrive at the 
intensity for the corresponding pixel, which is then displayed. 

[0044] It should be appreciated that while Figures 6 and 7 illustrate a one to one 
correspondence between texels and pixels, the invention is not limited to this relationship. 
That is, multiple texels may correspond to one pixel, e.g., minification, and alternatively 
15 one texel may correspond to multiple pixels, e.g., magnification. Additionally, these 
resampling operations may be filtered to avoid aliasing artifacts. 

[0045] Figure 8 is a simplified schematic diagram of the computing device capable of 
providing real time self-shadowing and self-interreflection for objects of an image in 
accordance with one embodiment of the invention. Computing device 160 includes chip 

20 161. Chip 161 includes central processing unit (CPU) 162, memory 169, and graphics 
processing unit (GPU) 168. In addition, self-shadowing and self-interreflection logic 164 
is in communication with CPU 162, memory 169, and GPU 168 over bus 166, Self 
shadowing and self interreflection logic 164 performs functionality to enable real-time 
presentation of lighting characteristics by applying the pre-processed lighting and self 

25 shadowing effects of a high resolution model to a low resolution model as described 
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herein. It should be appreciated that the pre-processed transfer function data describing a 
respective light field may be stored in memory 169 along with the texture map data. 
Computing device 160 includes storage medium 170, which is configured to alternatively 
store a texture map having texels that include data describing the light field as spherical 
harmonic coefficients or any other values associated with a suitable spherical basis 
function. Thus, through a rendering operation where the stored data is combined with the 
illumination data, an intensity of a pixel associated with a location corresponding to the 
stored data is calculated. It should be appreciated that the rendering is simplified to a dot 
product of the coefficients associated with the illumination data and the transfer function 
data, which in turn, defines a relatively fast calculation. Display screen 172 presents the 
image defined by computing device 160 using the rendered data. In one embodiment, 
computing device 160 may be a any suitable video game console, such as the 
"PLAYSTATION 2"® manufactured by Sony Computer Entertainment Inc. With 
respect to on-line gaming applications, computing device 160 may be a server. 
[0046] Figure 9 is a flow chart diagram of the method operations for rendering a low 
resolution image with high resolution lighting characteristics in accordance with one 
embodiment of the invention. The method initiates with operation 180 where a texture 
map associated with the image is generated. The texture map defines a plurality of 
texels. The method then advances to operation 182 where a value representing a lighting 
characteristic for each of the texels is calculated. In one embodiment, a spherical 
harmonic function may be applied as described above and in the article entitled 
"Spherical Harmonic Lighting: The Gritty Details," which has been incorporated by 
reference, to calculate the value representing the lighting characteristic. In another 
embodiment, the value is represented by a number of coefficients corresponding to a 
transfer function (T) described using directional basis functions. As described above, the 
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transfer function (T) accounts for the self shadows and self interreflection data. In other 
words, the transfer function represents the surface reflectance. Of course, a value 
associated with any suitable basis function may represent the lighting characteristics here, 
e.g., a wavelet transform, a spherical basis function, etc. The method then proceeds to 
5 operation 184 where the value calculated in operation 182 is stored on any suitable 
storage medium. In one embodiment, a value is calculated for each texel of the texture 
map, e.g., a center point of each texel. The method then moves to operation 186 where a 
coordinate space of the texture map is associated with a display screen coordinate space. 
It will be apparent to one skilled in the art that a number of readily-available techniques 
10 are provided in order to associate the coordinates of the texture space with the screen 
coordinate space. The method then advances to operation 188 where the image is 
rendered on a display screen. Here, the image appears as a high resolution image as 
described above. 

[0047] Figure 10 is a schematic diagram illustrating an on-line gaming application in 
15 accordance with one embodiment of the invention. Game console 260 is shown 
connected to server 262 through the Internet 254, using any wired or wireless connection. 
Server 262 may be one or more servers. Controller 259 is in communication with game 
console 260, which in turn is in communication with display screen 256. Game console 
260 includes storage 268 and is configured to accept computer readable media 202. It 
20 should be appreciated that the integrated circuit described with reference to Figure 8 may 
reside on server 262 or game console 260. Alternatively, software performing the 
functionality described above may similarly reside on server 262 or game console 260. 
[0048] In summary, the above described invention describes a method and a system for 
providing realistic shading for a video presentation in real time. The embodiments 
25 described above, pre-process the data using a basis function such as a wavelet or a 
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suitable omni-directional basis function, such as spherical harmonic lighting techniques. 
The pre-processed data, derived from the calculation of a transfer function, is associated 
with texture map data and stored in memory. In one embodiment, the transfer function 
yields data that is rotationally invariant. The stored data is then accessed and combined 
5 with an illumination value representing the incoming light, to define realistic shading for 
the image. In one embodiment, the pre-processed data is associated with a high 
resolution image, i.e., an image defined by thousands of polygons. Thus, when the 
shading defined by the pre-processed data is applied to a low resolution image presented 
on a display screen, the low resolution image appears to have a high resolution. 

10 [0049] It should be appreciated that the embodiments described herein may also apply to 
on-line gaming applications. That is, the embodiments described above may occur at a 
server that sends a video signal to multiple users over a distributed network, such as the 
Internet, as described with reference to Figure 10. The video signal defines images 
having realistic lighting characteristics that may be generated in real-time. 

15 [0050] With the above embodiments in mind, it should be understood that the invention 
may employ various computer-implemented operations involving data stored in computer 
systems. These operations include operations requiring physical manipulation of 
physical quantities. Usually, though not necessarily, these quantities take the form of 
electrical or magnetic signals capable of being stored, transferred, combined, compared, 

20 and otherwise manipulated. Further, the manipulations performed are often referred to in 
terms, such as producing, identifying, determining, or comparing. 

[0051] The above described invention may be practiced with other computer system 
configurations including hand-held devices, microprocessor systems, microprocessor- 
based or programmable consumer electronics, minicomputers, mainframe computers and 
25 the like. The invention may also be practiced in distributing computing environments 

SONYP025/MLG 20 Patent Application 



where tasks are performed by remote processing devices that are linked through a 
communications network. 

[0052] The invention can also be embodied as computer readable code on a computer 
readable medium. The computer readable medium is any data storage device that can 
5 store data which can be thereafter read by a computer system. Examples of the computer 
readable medium include hard drives, network attached storage (NAS), read-only 
memory, random-access memory, CD-ROMs, CD-Rs, CD-RWs, magnetic tapes, and 
other optical and non-optical data storage devices. The computer readable medium can 
also be distributed over a network coupled computer system so that the computer 

10 readable code is stored and executed in a distributed fashion. 

[0053] Although the foregoing invention has been described in some detail for purposes 
of clarity of understanding, it will be apparent that certain changes and modifications may 
be practiced within the scope of the appended claims. Accordingly, the present 
embodiments are to be considered as illustrative and not restrictive, and the invention is 

15 not to be limited to the details given herein, but may be modified within the scope and 
equivalents of the appended claims. In the claims, elements and/or steps do not imply 
any particular order of operation, unless explicitly stated in the claims. 

What is claimed is: 
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